Specter: Combining Music Information Retrieval with Sound Spatialization

Specter combines music information retrieval (MIR) with sound spatialization to provide a simple, yet versatile environment to experiment with sound spatialization for music composition and live performance. Through various interfaces and sensors, users may position sounds at arbitrary locations and trajectories in a three-dimensional plane. The system utilizes the JythonMusic environment for symbolic music processing, music information retrieval, and live audio manipulation. It also incorporates Iannix, a 3D graphical, open-source sequencer, for realtime generation, manipulation, and storing of sound trajectory scores. Finally, through Glaser, a sound manipulation instrument, Specter renders the various sounds in space. The system architecture supports different sound spatialization techniques including Ambisonics and Vector Based Amplitude Panning. Various interfaces are discussed, including a Kinect-based sensor system, a LeapMotion-based hand-tracking interface, and a smartphonebased OSC controller. Finally, we present Migrant, a music composition, which utilizes and demonstrates Specter’s ability to combine MIR techniques with sound spatialization through inexpensive, minimal hardware.